Design a CDN

Understanding the problem

A content delivery network, or CDN, is a fleet of caching servers spread across the world that sit between a website's visitors and the website's own origin server. When someone loads a page, the bytes come from a nearby cache instead of a single distant machine, which is what makes the page feel instant. A point of presence, usually shortened to PoP, is one of those physical locations: a data center in a city, packed with servers, peered directly with local internet providers. Cloudflare runs PoPs in hundreds of cities, and the whole product exists to answer a request without that request ever crossing an ocean to reach the origin.

This is the design candidates most expect from Cloudflare, because the company's stated mission is to help build a better Internet and the CDN is the product that most directly delivers on it. Every other edge design in this guide reuses the same skeleton. Users reach the nearest PoP by Anycast, the PoP answers from cache when it can, and only a cache miss talks to anything further away. Getting this one right means the DNS, rate-limiter, and DDoS designs mostly become variations on machinery you have already drawn.

The Cloudflare angle

A CDN is defined by the requests it answers WITHOUT going back to the origin, and at Cloudflare that fraction is the whole game. The naive mental model is a proxy that forwards traffic to the origin faster. The real product is a cache that answers the overwhelming majority of requests entirely at the edge, so the origin sees a tiny trickle of the true traffic. Interviewers here probe that gap hard, because it is the difference between a reverse proxy and a CDN. Lead by naming the cache hit ratio as the number the entire design optimizes: a well-tuned zone answers well above ninety percent of requests from the PoP. Every later choice about cache keys, TTLs, and tiered caching then visibly serves that one number. Naming it early shows the interviewer you know this is a CDN and not a load balancer with extra steps.

Functional requirements

Keep the scope on the request path and the freshness model, because that is where the design lives. A user asks for a URL, the nearest PoP returns the content, and if the PoP does not have a fresh copy it fetches one from the origin, stores it, and serves it. On top of that, the origin needs a way to say "this content changed, stop serving the old copy," which is cache invalidation. Everything else is a thread the interviewer can pull.

Non-functional requirements

Reason each target from the product. Latency is the reason the CDN exists. The goal is to answer a cache hit in single-digit milliseconds of edge processing, on top of the short network trip to a nearby PoP. That is only possible if the hit path never leaves the PoP. Read-heavy is the defining skew: the same popular objects are requested millions of times, and origin fetches are rare by comparison. So the design optimizes the read and treats a miss as the expensive exception. Availability beats strict freshness. A user is far better served a copy that is a few seconds stale than an error, so the edge should keep serving through an origin outage rather than fail. Freshness has real but bounded slack: a changed object should propagate within seconds of a purge, yet a page that is a minute stale under normal TTL expiry harms nobody.

One relaxation is worth banking now because it shapes everything after it. The cache is allowed to be eventually consistent with the origin. Two PoPs on opposite sides of the world can briefly hold different versions of the same object. That is fine as long as a purge converges them quickly and a TTL bounds how long any copy can drift. The CDN trades strict global consistency for availability and speed, and a purge plus a TTL are exactly where that staleness is bounded. That is a large part of what this prompt tests. Do not promise that every edge reflects every origin change the instant it happens.

• Latency: Single-digit ms hit. A cache hit is answered entirely inside the nearest PoP, so it costs edge processing plus a short local network trip. The origin round trip only happens on a miss.
• Read skew: Hits swamp misses. The same popular objects are fetched millions of times. The design optimizes the hit path hard and treats an origin fetch as a rare, expensive event to be shielded and coalesced.
• Freshness: Seconds after purge. A TTL bounds normal drift; a purge propagates an invalidation worldwide within seconds. Between those two mechanisms, staleness is bounded on purpose.
• Consistency: Availability first. PoPs are eventually consistent with the origin and with each other. Serving a slightly stale copy through an origin outage beats returning an error.

The set-up

Core entities

Name the objects before drawing. Four carry the design, and the one that does the real work is the cache entry, the stored copy of a response that a PoP serves without touching the origin.

The split that organizes everything is the hit path versus the miss path. On the hit path, a request arrives at a PoP, the PoP computes the cache key, finds a fresh cache entry, and returns it. Nothing leaves the PoP, and this is what almost every request does. On the miss path, the PoP has no fresh copy. It fetches from the origin through a shielded route, stores the response as a new cache entry under its key with a TTL, and serves it. The miss path is rare, expensive, and the thing tiered caching exists to protect, which is why the two paths get designed separately.

API

The external surface is small and familiar: it is just HTTP. A visitor issues an ordinary GET, and the CDN's cleverness is invisible to them. The interesting interfaces are internal and control-plane. Two matter: the origin fetch a PoP makes on a miss, and the purge call the customer uses to invalidate content across every PoP at once.

The call to dwell on is getFromOrigin. Everything expensive hides behind it. In a naive design every PoP that misses opens its own connection straight to the origin. A newly popular object with no cached copy anywhere hits the origin from hundreds of PoPs at once. Route that fetch through an upper-tier PoP instead. Now the origin serves the object once per region and the lower PoPs pull from the tier. Treating the origin fetch as something to funnel and coalesce, rather than something each PoP does alone, is the mental shift the rest of the design hangs on.

High-level design

Draw the smallest version that shows both paths, then go deep on the cache key, the freshness model, and tiered caching, because those are the parts that make this a CDN. The hit path is a local lookup. A user's packet lands at the nearest PoP by Anycast, and the edge server computes the cache key, finds a fresh cache entry, and returns the bytes without touching anything else. The miss path runs alongside it. When the PoP has no fresh copy, it asks an upper-tier PoP, which either serves its own cached copy or fetches once from the origin. The object then flows back down and gets stored at each tier on the way.

Color marks the role. Blue is the user requesting a URL. Violet is the edge servers, the lower-tier PoP that faces the user and the upper-tier PoP that shields the origin. Amber is the purge fan-out that carries invalidations to every PoP. Green is the caches and the origin, the lower PoP cache, the upper-tier cache, and the customer origin behind them.

1) Route to the nearest PoP with Anycast. A user's request first has to reach a server, and the routing is done by the network itself, not by the application. Every PoP announces the same Anycast IP address over BGP, the protocol internet routers use to decide paths, so a packet is delivered to whichever PoP is fewest network hops away. A user in Frankfurt lands on the Frankfurt PoP and a user in Tokyo lands on Tokyo, for the same URL, with no DNS trickery or client logic. This is the step that puts a cache within a few milliseconds of every user on earth. What it punts is what the PoP does once the packet arrives, which the cache lookup answers next.

2) Answer from the local cache on a hit. The lower PoP computes the cache key from the request, usually hostname plus path, and looks for a fresh cache entry. When it finds one, it returns the stored bytes and headers immediately, tagging the response a HIT, and nothing leaves the PoP. This is what the overwhelming majority of requests do, and it is why a well-tuned zone answers well above ninety percent of traffic in single-digit milliseconds of edge work. The hottest objects are held in memory; the long tail sits on local disk. What is deliberately kept off this path is any origin contact at all, which only the miss path incurs, and how the entry got there and stays fresh, which the deep dives own.

3) Funnel a miss through the upper tier. When the lower PoP has no fresh copy, it does not open a connection to the origin. It asks its upper-tier PoP, the region's origin shield. If the tier has the object cached, it serves it down and the origin is never touched. If the tier also misses, it makes a single fetch to the origin, stores the result, and passes it down. The lower PoP then stores its own copy under the cache key, with a TTL derived from the origin's directives. So a brand-new popular object costs the origin exactly one fetch per region no matter how many lower PoPs and users are clamoring for it. What this step punts is what happens when many requests miss the same key at the same instant, which the deep dive on cache stampede handles.

4) Keep it fresh with TTL and purge. A cache entry carries an expiry from its TTL, so it naturally goes stale and gets revalidated against the origin after that window. A cheap conditional request with the object's ETag usually comes back "not modified," refreshing the copy without moving bytes. That handles routine drift. Some content changes on the customer's schedule rather than a clock. For that, the purge control plane fans an invalidation out to every PoP within seconds, by URL or by tag, so a deploy or a price change is reflected globally almost immediately. Between the TTL bounding passive drift and purge handling active change, staleness stays bounded. The deep dives take these apart, starting with the cache key that decides what counts as the same object.

Back-of-envelope: sizing the edge

A few numbers set the shape of the whole design. Start from footprint. Take a network on the order of 300 cities, so a few hundred PoPs, each with racks of cache servers. Suppose the network as a whole serves tens of millions of requests per second at peak across all customers, call it 50 million RPS to work with a round figure. Spread over a few hundred PoPs that is on the order of 100 to 200 thousand requests per second at a busy PoP. A rack of servers answers that comfortably, because a cache hit is cheap. The point of the arithmetic is that the request rate is enormous but the per-PoP rate is ordinary once Anycast has spread it, so no single machine is heroic.

The number that decides everything downstream is the cache hit ratio. At a 95 percent hit ratio, out of that 50 million RPS only about 2.5 million requests per second are misses, and tiered caching collapses those further so the origins collectively see far less. Push the hit ratio to 98 percent and origin-bound traffic drops to about 1 million RPS. Three points of hit ratio cuts origin load by 2.5x, which is exactly why cache key design and TTL tuning get so much attention. Egress tells the same story. A single popular 4K video re-fetched from origin for every viewer would be ruinous. Cached at the edge, it is pulled from origin once and served millions of times, so the CDN turns origin egress into edge egress and the origin's bandwidth bill nearly vanishes. Storage per PoP is modest: hold the hot working set, a few tens of terabytes of the most-requested objects, and let the cold long tail miss to the tier.

• PoPs: ~300 cities. A few hundred edge data centers sharing one Anycast address. The unit of both caching and failover; losing one reroutes its users to the next-nearest with no DNS change.
• Edge RPS: ~50M total, ~150K / PoP. Tens of millions of requests a second network-wide, but Anycast spreads it. A busy PoP handles a couple hundred thousand, an ordinary load for a cache.
• Cache hit ratio: ~95 to 98%. The number the whole design optimizes. Each point of hit ratio is a large cut in origin traffic; 95 to 98 percent roughly halves origin-bound RPS.
• Origin egress: Fetched once, served millions. A hot object is pulled from origin one time per region and served from the edge forever after. The origin's bandwidth is a rounding error next to edge egress.

Technology choices

Every box maps to a real category, and the Cloudflare-specific constraint is that these run on the company's own hardware in its own PoPs, not on a managed cloud. Name the popular pick, say why that kind of tool fits, and be ready for the failure follow-up, because that is where the round goes.

• Routing is Anycast over BGP, the same address announced from every PoP so the internet delivers each user to the nearest one. Anycast fits because it needs no central load balancer and no per-user DNS decision. The routing table does the work, and adding or draining a PoP is a BGP announcement rather than a config push to millions of clients. The failure to answer for is a PoP going dark: its BGP announcement is withdrawn, routers converge on the next-nearest PoP within seconds, and users see a brief reroute rather than an outage. The subtlety is that a long-lived connection can occasionally flap to a different PoP mid-session if routes change, which is why the cache is shared-nothing per PoP and requests are idempotent.
• The edge cache server is a reverse-proxy cache, the role NGINX plays and the job Cloudflare's own edge stack does in-house. This kind of server fits because it has to do three cheap things per request at enormous rate: compute a cache key, look up an entry, and stream bytes. It keeps the hot working set in memory and spills the long tail to local NVMe disk, so a hit is a memory or fast-disk read, never a network call. The failure to reason about is a single cache server dying inside a PoP. The PoP has many, and requests hash across them. A lost server means its share of keys miss to the tier until traffic rebalances, degrading hit ratio slightly rather than dropping requests.
• The origin shield is tiered caching: designated upper-tier PoPs that every lower PoP routes misses through, so the origin sees at most one fetch per object per region. A tier fits because the alternative, every PoP fetching the origin independently, turns one newly popular object into hundreds of simultaneous origin hits. The failure to plan for is the upper tier itself missing under a thundering herd, when many lower PoPs request the same uncached key at once. The fix is request coalescing at the tier. The first miss fetches the origin, and the rest wait on that single in-flight fetch instead of piling more load onto the origin.
• The purge control plane is a fan-out over a fast distribution bus, the role a system like Cloudflare's Quicksilver plays for pushing config and invalidations to every PoP in seconds. This fits because a purge must reach hundreds of PoPs quickly and reliably; a slow or lossy purge means users see stale content past what the customer expects. The failure to name is a PoP that was offline during a purge. On rejoin it must replay missed invalidations from the bus before it serves, so it never comes back holding content the rest of the world already dropped. Tag purge is implemented by storing each entry's tags and letting a tag invalidation match many keys at once.
• The cache storage is the PoP's own memory plus local NVMe disks, deliberately not a managed object store like S3. Local storage fits because the entire latency win comes from the bytes being feet away from the request, not a network hop away, and because Cloudflare runs its own metal at the edge. The failure to handle is eviction pressure: the working set exceeds RAM and disk, so a least-recently-used policy drops cold objects, which simply miss to the tier next time they are wanted. Freshness on disk is enforced by the stored expiry: an entry past its TTL is revalidated with a conditional request before it is served, so stale bytes are never returned as fresh.

Name the brand, defend the category. The genuinely Cloudflare pieces here are Anycast-everywhere routing, tiered caching with an upper-tier origin shield, and a config-and-purge bus that reaches every PoP in seconds, all on owned hardware. The reverse-proxy cache and the LRU eviction are standard components doing a familiar job. The through-line comes from the requirements. Answer from the nearest edge, keep the origin nearly idle, and let a copy be briefly stale in exchange for staying fast and available. If an interviewer here asks how you would host this, the wrong answer names a cloud service; the right answer names PoPs, local storage, and a distribution bus you run yourself.

Potential deep dives

The signal lives here. A Cloudflare interviewer will push on four things. How the cache key is built and why key design decides hit ratio. How a TTL-plus-purge model keeps content fresh without a strong-consistency crutch. How tiered caching survives a cache-miss storm. And what the edge does when the origin is down. Take them in that order.

1) The cache key: what counts as the same object

The cache key is the string a PoP computes from a request to decide whether it already has the answer. Every point of hit ratio, and every subtle correctness bug, traces back to how this key is built, so it is the first thing a strong interviewer digs into. Walk it from the naive version up.

The Cloudflare angle

At Cloudflare, cache key design is a product surface customers configure, so the interviewer expects you to reason about it as a knob. The default key is host plus path, but zones override it. They can include or ignore specific query strings, fold in a device type, key on a cookie for logged-in variants, or explicitly cache a URL the origin marked private. Show that you know the key is where a customer trades hit ratio against correctness, and that the safe default is to strip aggressively and add back only what the origin genuinely varies on. Name utm and fbclid as the canonical parameters to strip, and a raw User-Agent as the canonical thing never to key on directly. That detail shows real CDN tuning behind the answer.

2) Freshness: TTL, revalidation, and instant purge

A cache is only useful if it is also correct, and correctness means not serving badly stale content. Cloudflare gives customers two complementary freshness tools, a clock-based TTL and an event-based purge. The design question is how they combine to keep staleness bounded without forcing strong consistency on a globally distributed cache.

TTL is the passive floor. When the origin returns an object it attaches a Cache-Control directive, and the edge stores the object with an expiry that far ahead. Until then the copy is served as a fresh hit. After expiry the object is kept. The edge revalidates it with a conditional request carrying the stored ETag, and the origin almost always answers "not modified," which refreshes the expiry without transferring a single byte. So a long TTL on an image that rarely changes means near-zero origin traffic, while a short TTL on a news homepage means frequent cheap revalidations. The TTL alone bounds how stale any copy can be under normal operation, but it is a blunt instrument: it cannot reflect a change the instant it happens.

Purge is the active override, and it is where the "instant global invalidation" claim gets tested. When a customer deploys new content or corrects a mistake, they call purge. The invalidation fans out over the distribution bus to every PoP within seconds, dropping or marking-stale the matching entries so the next request refetches. Single-URL purge is the precise case. Tag purge is the powerful one. The origin labels related objects with a cache tag, say all pages showing a given product. One tag purge then invalidates every entry carrying that tag across the whole network at once, which is how a price change ripples out without listing thousands of URLs. Full-zone purge nukes everything, used sparingly because it forces a cold cache and a miss storm.

The Cloudflare angle

The reason Cloudflare can promise fast global purge is the same config-distribution bus that ships routing and security rules to every PoP, so treat purge as a broadcast problem. A purge is just another message on the bus that carries edge configuration everywhere in seconds. The cache subscribes to invalidations the way it subscribes to config changes. State the tradeoff plainly: TTL bounds passive drift so most content needs no purge at all, and purge handles the moments when the customer needs a change reflected faster than the TTL would allow. A candidate who reaches for strong per-object consistency across 300 PoPs is optimizing the wrong axis and would wreck the availability the product depends on. The senior move is to name where staleness lives, TTL for the clock and purge for the event, and bound both, rather than trying to make every edge agree with the origin instantly.

3) Tiered caching and the cache-miss storm

The scariest moment for a CDN is a popular object that is not cached anywhere: a new video drops, a viral image posts, a cache was just purged. Suddenly hundreds of PoPs and millions of users all want a key that no edge holds, and the origin is about to get the full flood it was supposed to be shielded from. Tiered caching and request coalescing are the two mechanisms that stop that flood.

Tiered caching adds a middle layer. Instead of every lower PoP fetching the origin on a miss, each region routes its misses through a designated upper-tier PoP, and only that tier talks to the origin. So fifty lower PoPs that all miss the same object generate fifty requests to the tier but, because the tier caches, at most one request to the origin. The origin's exposure drops from "one hit per missing PoP" to "one hit per region," which is the difference between a small origin surviving a viral moment and falling over. The tier is chosen for network proximity to the origin, so its own fetch is fast. The objects flow back down and populate every layer on the way so the next request is a hit at the edge.

Request coalescing handles the same-instant version of the problem within a node. When many requests for the same uncached key arrive at a PoP or at the tier in the same moment, the cache does not fire one origin fetch per request. The first miss becomes the single in-flight fetch, and every other request for that key parks and waits on it. When the bytes arrive, they all get served from the one fetch and the entry is now cached. Without coalescing, tiered caching alone would still let a thundering herd of simultaneous misses at the tier each open its own origin connection. With both, a brand-new global-scale object costs the origin essentially one fetch, then never touches it again until the TTL expires.

The Cloudflare angle

Tiered caching is the concrete answer to the "no managed cloud, host it yourself" constraint Cloudflare interviewers apply, so build the shield out of your own PoPs. The wrong instinct under load is to reach for a managed queue or an origin behind a cloud auto-scaler. In the Cloudflare answer the network shields the origin structurally, using a hierarchy of PoPs the company already runs. Draw the two tiers explicitly and put request coalescing at each tier. Then reason at the real magnitude: a network of a few hundred PoPs collapses to a handful of regional tiers collapses to one origin fetch. That is the anycast-and-tiers machinery from the high-level design, now doing its most important job. Name it as the origin's protection under a viral spike; a senior CDN round looks for that depth.

4) When the origin is down: serving stale to stay up

An origin will eventually be unreachable, from a deploy gone wrong, a regional outage, or an overload. The question is what the edge does at that moment. The CDN's answer is the reason customers put it in front of their origin in the first place: keep the site up on cached content rather than pass the failure through to users.

The mechanism is serving stale on error. When an entry's TTL has expired and the edge tries to revalidate but the origin returns an error or times out, the edge does not surface that error to the user. It serves the last good cached copy, stale but working, and keeps retrying the origin in the background. From the visitor's point of view the site stays fast and functional through an origin outage; the only cost is that content is frozen at its last-known-good version until the origin recovers. This is a deliberate availability-over-freshness choice, the relaxation banked in the requirements. It is configurable per zone because some content (a stale bank balance) must never be served old while other content (a marketing page) obviously should.

Two related failures round this out. A single PoP failing is handled by Anycast: its BGP announcement is withdrawn, and routers converge on the next-nearest PoP in seconds. That PoP simply has a colder cache for the rerouted users until it warms, so the failure shows up as a brief hit-ratio dip rather than an outage. A missed purge during a PoP's downtime is handled by reconciliation. When the PoP rejoins the distribution bus, it replays the invalidations it missed before it starts serving, so it never comes back holding content the rest of the network already dropped. Across all three, the same principle from the requirements shows up at the failure layer: stay available and fast, and let freshness be the thing that flexes.

What is expected at each level

How much you need to land scales with the level. Cloudflare is deliberately opaque about its ladder, which internally runs on P-bands rather than the L-numbers other companies expose. So the bars below map roughly to three tiers: an early-career engineer, a senior engineer (commonly the SSE title in candidate reports), and a staff-level scope. Treat the mapping as approximate. System design carries more weight the more senior the loop, and the edge-infrastructure flavor is graded harder here than at a generic web company.

Early-career engineer

Get the request path right and defend it. Frame the CDN as a cache that answers most requests at the nearest PoP. Draw the hit path staying local and the miss path going to the origin, and explain why the cache hit ratio is the number that matters. Name Anycast as how a user reaches the closest PoP, and describe TTL and purge as the two ways content stays fresh. The interviewer will drive the harder parts. The bar is a coherent skeleton, route to PoP, hit or miss, fetch and cache on a miss, keep fresh, that you stand behind with a nudge or two.

Senior engineer

Depth carries it, and the cache key and the freshness model are where you earn the level. Raise cache key design unprompted, explain why keying on the raw URL wrecks hit ratio and how normalization and bounded Vary variants fix it. Lay out TTL for passive drift and purge for active change, and state the eventual-consistency tradeoff plainly rather than reaching for strong consistency across PoPs. Bring up tiered caching and request coalescing as the origin's protection under a viral spike, and reason at the real magnitude of a few hundred PoPs.

The bar is a near-complete design where you found the hard parts, the key-cardinality trap and the cache-miss storm, on your own. Most senior candidates are sorted right here.

Staff engineer

Lead across the whole system. Treat serving stale on error as a per-zone availability choice. Then reason about how the edge degrades rather than breaks when the origin is down, a PoP drops, or a purge is missed during downtime. Weigh cache key normalization against correctness, naming exactly where over-stripping causes two responses to collide. Hold the line on the no-managed-cloud constraint, building the origin shield and the purge fan-out out of Cloudflare's own PoPs and distribution bus rather than a cloud primitive. Place the boundaries cleanly: where the CDN ends and the DDoS, DNS, and Workers designs begin, so you are not redesigning the whole edge in one round. The bar is that you drive the room while the interviewer pushes back.